TWO-DIMENSIONAL SLICES OF 
NON-PSEUDOCONVEX OPEN SETS 



NIKOLAI NIKOLOV AND PETER PFLUG 

Abstract. Let I? be a non-pseudoconvex open set in C 3 and S be 
the union of all two-dimensional planes with non-empty and non- 
pseudoconvex intersection with D. Sufficient conditions are given 
for C 3 \ S to belong to a complex line. Moreover, in the C 2 -smooth 
case, it is shown that S = C'\ 



It is well known (see [2J, [3]) that a open set D C C n is pseudo- 
convex if and only if D fl P is pseudoconvex (or empty) for every 
two-dimensional plane P C C n . Hence, pseudoconvexity is, in fact, 
a two-dimensional phenomenon. Let us denote by S = S(D) the set of 
all points oeC™ such that there exists such a two-dimensional plane 
P, a G P, such that P fl D is non-pseudoconvex. Points which do not 
belong to S are called exceptional points for D. In the following 
results are shown: 

(1) if D C C n , n > 3, is not pseudoconvex, then all exceptional 
points belong to one complex hyperplane; 

(2) if D is C 2 -smooth and non-pseudoconvex near a boundary point, 
then all exceptional points belong to a three-codimensional plane 

(3) there is an unbounded domain in C 3 with real-analytic bound- 
ary except one point, the origin, which has exactly one excep- 
tional point, namely the origin; 

(4) if D = D \ I C C 3 , where D is a pseudoconvex open set and I 
a complex line intersecting D , then S = C 3 \ (/ \ D); 

(5) if D is as before and if Zi, I2 are two different complex lines with 
Ij fl D ^ 0, then C 3 \ S = (h n h) \ D. 

Looking more carefully at the example (3) which is given as 

D = {z G C 3 : |z 3 | 2 < N 2 + N 2 < 4N 2 }, 
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we see that the restriction of the Hessian (resp. the Levi form) of r(z) = 
\ z 3 1 2 — I z i 1 2 — \ z 2 1 2 to the complex tangent hyperplane T p at any non-zero 
p with r(p) = has eigenvalues 0, 0, —1, —1 (resp. and —1). Moreover, 
if l p denotes the complex line through and p, then T p \ l p C D and 
l p C 3D. 

So, it is natural to study the Hessian as in the C-convex case because 
here we play with intersections with complex lines. 

The following proposition will play a key role in the next considera- 
tions. 

Proposition 1. Assume that D is an open set in C 3 such that G 3D 
and D D G H D 3 for some e > (B 6 := {A G C : |A| < e}), wher£\ 

G = {z G C 3 : > 2 Re ;z 3 + (Im^) 2 + \ Zl \ 2 - \z 2 \ 2 }. 

If S ^ C 3 , then here exists a complex line I G C 2 x {0} C C 3 through 
such that C 2 x{0}\IcD near 0, I C 3D near 0, and SdC 3 \I. 

Proof. Let a G C 3 . If a% ^ 0, then the intersection of D with the com- 
plex hyperplane through the point 0, a, and (0, 1, 0) is non-pseudoconvex; 
use that Levi form of G at is negative and apply the Kontinuitatsatz. 
Hence S D {z G C 3 : z 3 ^ 0}. 

Fix a S j$ a = (a 1; a 2 , 0) 7^ and put I = <Ca. It will be enough to 
show that 

(1) D' x {0} := D n {2 G C 3 : z 3 = 0} D (d 2 /2 \ /) x {0}. 

Indeed, suppose that there is a 6 = (61,62,0) G" 5 U /. It follows by 
(HJ that £>' D D 2 /2 \{zeC 2 :M 2 = 6 2 Zi}. Hence 0^Z)'D D 2 /2 \ {0}. 
In particular, D' is non-pseudoconvex which contradicts the fact that 
a <£ S. 

To prove ©, let (0, 0) ^ (p, q) G C 2 , P = P{p, q) := {z G C 3 : z 3 = 
p(zi - d) + g(^ 2 - a 2 )}, and P £ = P n D 3 . Then 

D n P D G n P £ D G P e : |^i + p\ 2 < \z 2 - q\ 2 + s}, 

where 

s = 2Re(p ai + qa 2 ) + |p| 2 (l - 2(|ai| + e) 2 ) - \q\ 2 (l + 2(|a 2 | + e) 2 ). 
Thus we may find a positive e' < e such that if (p, q) G © 2 /, then 
D D D e := {z £ P : \z\ + p\ 2 < \z 2 - q\ 2 + s' , {z u z 2 ) G D 2 }, 

1 For an arbitrary non-pseudoconvex set D in C 3 , after an afnne change of co- 
ordinates, we may assume the same but with an extra summand c(Imz2) 2 , c < 2 
(cf. [3]). So, after a quadratic polynomial automorphism of C 3 , we have the original 
domain G locally inscribed in D. 
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where 



s' = 2Re(pa l + qa 2 ) - \p\ 3/2 - \q\ 3/2 . 
Take now a point z° = (zi,z%,0) G D^ 2 with a\z\ 7^ fl^z?- Assume 
that z 2 7^ a 2 (the case z® 7^ ai is similar). It is easy to check that there 
is rj < e'/2 such that for any p 7^ 0, \p\ < 77, with argp = arg ( ai2 g_ a2Zl J 



and q = we have s' > and (zj + p, ;z 2 — g) G 2 , 2 . Fi x sucn 

a pair (p, g) and let P = P(p, q) be the corresponding plane. Observe 
that z° G P. Then 

D £ D D £ := {z <E P : \z\ + p| 2 < |s a - q\ 2 + s' , (z 1 + p, z 2 - q) G D 2 /2 }. 
Then the envelope of holomorphy of D £ is 



as it is known from the study of Reinhardt domains. Since D D P is 
pseudoconvex (because a ^ S 1 ), it follows that z° G if C -D fl P which 
proves ([1]). 

It remains to show that I C D near 0. We know that D x {0} := 
D fl {2 G C 3 : 2:3 = 0} fl B^, 2 is a pseudoconvex open set and D 2 ^ 2 \ / C 

L> C (D 2 )*. By the claim below, D = B 2 /2 \ I. Therefore it follows that 
(/ n © 2 /2 ) x {0} C 3D. 

Claim. If I = Ca, \\a\\ = 1, is a complex line in C 2 and G C D 2 is a 
pseudoconvex open set containing IB) 2 \ I, then G = O 2 \ I. 

Indeed, first note that G is a domain, since if D 2 3 Xa, then for any 
sufficiently small t > we have that Xa + £(— a2,ai) G D 2 \ /, where 
a = (01,02). Obviously, we may assume that |a 2 | < |ai| (otherwise 
interchange the coordinates). Now suppose that G 7^ D 2 \ /. Then 
there exists a point Ao<2 G G and so we have that @ 

V e := © £ (A ai) x O e (A a 2 ) C G 

for a small e > 0. Moreover, simple calculation gives that 

(D e (A ai) x D) n £ C V e . 

Therefore, U e := D e (Aofli) xDcG. Fix an arbitrary r < 1 such that 
D e (A ai) C D r . Then 



defines a holomorphic function on W r := H> r x D(i+ r )/2> which coincides 
with / on D £ (A ai) x B( 1+r )/ 2 . Noting that W r nG is connected, we get 

2 D e (c) will always denote the disc with center c € C and radius e. 




H = {z E P : (z 1 +p,z 2 - q) EB 2 £/2 }, 
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/ = F on W r n G. With r/1 we finally get a holomorphic extension 
of / to the whole bidisc; a contradiction. □ 

We like to mention that the claim before remains true for any com- 
plex (not necessarily passing through the origin) line intersecting the 
bidisc (with a similar proof). 

Next we state the first main result. 

Theorem 2. Let D be an open set in C 3 and p one of its boundary 
points. Assume that D has a C 1 - defining function r near p (i.e. its 
gradient at p is non-zero ) which is twice differentiable at p and that the 
Hessian of r at p restricted on the complex tangent hyperplane T p at p 
has two negative eigenvalues and one non-positive eigenvalue. 

If S ^ C 3 , then there exists a complex line I through p such that 
T p \l C D near p, I C dD near p and S D C 3 \ I. 

Moreover, for any point b$.lorb = p, there exists a two-dimensional 
complex plane Pb C C 3 through p and b such that the intersection DC\Pb 
is non-pseudoconvex (so p G S). On the other hand, this fails to be true 
ifbel and b ^ p. 

In particular, the Hessian (and hence the Levi form) at p vanishes 
along I; therefore, the Hessian (resp. the Levi form) at p restricted to 
T p has two (resp. one) negative and two (resp. one) zero eigenvalues. 

Before presenting the proof we like to mention that the used condi- 
tions for the eigenvalues are optimal. Indeed, there are pseudoconvex 
examples with one negative and two zero eigenvalues, as well as with 
two negative eigenvalues; for example, 

{z G C 3 : Re^i + 2(Rez 2 ) 2 - (Im^ 2 ) 2 < 0} and 

{z G C 3 : Re z x + 2(Re z 2 f - (Im z 2 ) 2 + 2(Re z 3 ) 2 - (Im z 3 ) 2 < 0}. 

Proof. We may assume that p = and To = {z G C 3 : z 3 = 0}. Take a 
point a G C 3 . If a% ^ 0, then the beginning of the proof of Proposition [1] 
provides a respective complex hyperplane through a (and even through 
0) (because the Levi form at has a negative eigenvalue; see the ar- 
guments below). Let a 3 = 0. Let t>i, v 2 , v 3 , t> 4 be the (orthonormal) 
eigenvectors of respective matrix to the restriction H of the Hessian 
at on z 3 = which correspond to the eigenvalues Ai < 0, A 2 < 

0, A 3 < 0, A 4 . Consider the complex line G s C L := span(t> 1; t> 2 , v 3 ) 
(if t> 4 = (tx,t 2 ) G C 2 , then s = {z G C 2 : 1 X Z\ = -t 2 z 2 }). We may 
assume that s is the zi-line. If H is negative on s, then we may use 
Proposition [1] (we have the respective inscribed model G). Otherwise, 
A3 = and, after a rotation in s, we may assume that span(w 3 ) = Im^!, 

1. e. H = H(z x ,z 2 ) does not depend on Im^. We shall show that 
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S = C 3 . After a rotation in the z 2 -line, we may assume that L contains 
the Imz 2 -axis. Then H(Rez 1: lmz 1: 0, lmz 2 ) < Ci(| Re^ 2 + | lmz 2 \ 2 ), 
where c\ = max(Ai,A 2 ). Fix a c 2 > C\. Then we may find a c 3 > 1 
such that 



So, for any e > there is a bidisc D 2 such that 

D 3 D {,2 G © 2 : c 3 (Rez 2 ) 2 + c 2 (lmz 2 ) 2 < -c 2 (Re Zl ) 2 - ^Im^) 2 }. 

After a dilation of coordinates, we may find c > such that for any 
e > there is a bidisc IB) 2 - with 

D 3 D {z G D 2 : (Rez 2 ) 2 - c(Im^) 2 < (Re^i) 2 - ^(Im^i) 2 }. 

The following lemma shows that D 3 is non-pseudoconvex which com- 
pletes the proof. 

Lemma 3. Let 

G := {z = (xi + iyi,x 2 + iy 2 ) eB> 2 : xj- cy\ < x\ - ey 2 } 

with c > max(e, 0). Then any open set D C C 2 satisfying G C D and 
G dD is non-pseudoconvex. 

Proof. Note that, decreasing c and increasing e, we shrink G. So, we 
may assume that either 0<e<c<lorl<£<c. 

Let first < e < c < 1. Choose 1 < p < q such that e = ^j- and 

c = Moreover, let $ : C 2 — > C 2 be the following biholomorphic 



mapping: &(z) := (y^p^i, y^r-^)- Then a simple calculation shows 
that 



$(£>) D = {w G © /^rxD ^ : Re(w 2 -u> 2 ) < HVp-KI'M- 



So, we may suppose that 

$(£>) D G = G D 2 ^ : Re^ 2 , - wf) < {w^/p - \w 2 \ 2 /q}, 

where 77 is suitably chosen. 

Put S t = {w G D 2 : w\-w\ = t} andTt = {w e dD v : w^-wf = t}, 
where t < (St ^ for t ~ 0). Then the triangle inequality shows 
that S t (Z G C $(£>) and U«-*«r, 2 T t ^ <5 C $(£>). Moreover, note 
that $ 9 0) ^ (0,0) G Recall that by the maxi- 

mum principle on analytic hypersurfaces we have ||/||s t uT t = \\f\\T t for 



D 3 x {0} := D n {z G C 3 : z 3 = 0} D 
({z G C 2 : c 3 (Rez 2 ) 2 +c 2 (Imz 2 ) 2 < -c 2 (Re ^f+oQ Im^| 2 )}) x{0} 
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any function / holomorphic on $(-D) (—t <C 1). Applying the Konti- 
nuitatssatz implies that $(-D) cannot be pseudoconvex. Therefore, D 
is non-pseudoconvex. 

It remains to note that the case 1 < e < c reduces to the case 
considered before by the biholomorhic mapping ^f(z) = (iy/ez2,i>/czi) 
(and replacing D 2 by \I/(0 2 )). □ 

□ 

Remark 1. The assumption c > max(e, 0) in Lemma [3] is essential; 
for example, if £ > c = 1, then G is pseudoconvex, and if c < 0, then 
G C D, x D and the last domain is pseudoconvex. 

Note that Proposition 1 also implies the following 

Proposition 4. Let D and r be as in Theorem^ and assume that the 
restriction of the Hessian of r at p on a complex line in T p has two 
negative eigenvalues. Then the same conclusions hold as in Theorem 

m 

Indeed, we may assume that p = and the Hessian is negative on 
the z 2 -line. Then, after a dilation of coordinates, we have the inscribed 
model G as in Proposition [TJ 

Finally, we have the following result. 

Proposition 5. Let D, r andT p be as in Theorem^ Assume that the 
Levi form of r at p restricted on T p has two negative eigenvalues. Then 
S = C 3 . Moreover, for any point a G C 3 there exists a two-dimensional 
plane P C C 3 containing p and a, such that the intersection D D P is 
non-pseudoconvex for every two-dimensional plane P C C n ; 

Proof. We may assume that p = and T = {z G C 3 : z% = 0}. If 
a ^ To, then a G S (as before). To see that a G S if a G T , it 
suffices to show that D' — D fl To (considered as two-dimensional set) 
is non-pseudoconvex. Note that 

D'neJ = {z6Dj: f{z) = r(z,0) = cReq{z) + £{z) + o{\z\ 2 ) < 0}, 

where £ denotes the Levi form of f at 0, c G {0,1}, and q is a (holo- 
morphic) homogenous polynomial of order 2. The case c = follows 
immediately by Hartog's theorem. So let c = 1. Then there exist 
e G (0,77) and d > such that r(z) < Req(z) — d\\ z\\ 2 , z G D 2 . Put 
A t = {z G ID 2 : q(z) = t}. Then there exist a sequence (aj) with 
dj — > such that tj := q(aj) = Reg (a-,) < and A tj is an analytic 
hypersurface (use that q is an open mapping). Put Sj := A tj and 
Tj := A t . fl dH) 2 . Then one may find a k such that \JJL k Sj C D' and 
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\JjLkTj <s D' (use the upper estimate for r from above). Suppose 
now that D' is pseudoconvex. Then the Kontinuitatsatz implies (sim- 
ilar as in the proof of Lemma [3]) that \J**=k ^ D' . On the other 
hand, we know that Sj 3 dj — > G dD'; a contradiction. Hence D' is 
non-pseudoconvex. □ 

In contrast to (3) we have the following general result. 

Theorem 6. Let D C C™ be a C 1 -smooth open set having a C 2 -smooth 
non-pseudoconvex boundary point. If the set of non C 2 -smooth bound- 
ary points of D has zero (2n — 2)-Hausdorff measure, then D has no 
exceptional points. 

In particular, we may formulate the following pseudoconvex test for 
C 2 -smooth open sets. 

Corollary 1. A C 2 -smooth open set D in C" is pseudoconvex if and 
only if it allows one exceptional point. 

Proof of Theorem® Put d g D the set of all C 2 -smooth boundary points 
of D. Then we find an open neighborhood U of d g D and a defining 
C 2 -function r for dD along d g D on U with || gradr|| = 1 on d g D. 

Suppose now that there exists a point a G C n such that for every 
two-dimensional plane P passing through a the intersection P fl D is 
either empty or pseudoconvex, i.e. a is an exceptional point for D. We 
may assume that a = (use translation). 

Now, take an arbitrary point b e d g D, b ^ 0, such that there exists at 
least one complex tangent vector X G T^(dD) (i.e. Y^j=i ~§r(b)Xj = 0) 
such that Cr{b; X) < 0. 

Denote by s(z) = s r (z), z G d g D, the smallest eigenvalue of the Levi 
form of r, restricted to the complex tangent hyperplane T^(dD), and 
put 

t z := inf{t G (0, 1] : VV e[M] : tz G d g D and s(rz) = s(z)t < 0}. 

If tb > 0, then we claim that £&& G dD \ d g D. Indeed, we may assume 
that b = (bi, 0, . . . , 0) with b\ = Re&i < (use rotation). Suppose now 
the contrary, namely that fob G d g D. 

So we have ^ c =: fob G d g D and s(c) = s(b)/t . Since r(tc) = 
for t belonging to some interval [1, 1 + e) we get dr / dx\(tc) = for 
these t's; in particular, dr/dxi(c) = 0. 

From now we identify a complex point z = (xi+iyi, . . . , x n +iy n ) with 
its real counterpart (xi, • • • , X2 n ) (i-e. yj = x n+ j for j = 1, . . . , n). Then 
we may solve the equation r = near c w.r.t., say the k-th coordinate 
(k 7^ 1). So we have an open £-cube Q$ with center (ci,0, • • • , 0) G 
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IR 2 ™ 1 and an interval / := (—77,77) such that x G (Q x I) fl <9.D if 
and only if x fc = /(5), x E Q. We know that /(£i,0) = as long 
£1 G (ci - 6,ci]. 

By assumption (recall that s(c) < 0), there is 5' < 5 a open <5'- 
cube K = Q«5' C IR 2 ™ -1 C Q with center (c ± ,0, ■■■ , 0) such that all 
boundary points (x,f(x)), x G V, have a Levi form whose restriction 
to the corresponding complex tangent hyperplane is not positive semi- 
definite. Therefore, their complex tangent hyperplanes pass through 
the origin; the same for the real tangent hyperplane means that 

^ dx~-^ Xj = * eV ' 

Using the Euler differential equation we see that 

/(fx) = tf(x), xeV,te I(x) := (1 - v(x), 1 + n(x)), 

where I(x)x C V. In particular, we have for all boundary points 
x = (x, f(x)) sitting in V x I that I(x)x G dD. 

Take as a new defining function of D near the point c the following 
one: p(z) := x k -f(x 1 , x k - U x^+i, ■ ■ ■ , x n , y u . . . , y n ). Observe now, 
using homogeneity of / near (ci,0), that its gradient is constant near 
c along the boundary of D. Call its norm by a. Then, again by 
homogeneity of /, we see that for t £ (t — i],t Q }, < i] small, we have 
(to - 77, 1] C dD and 

s(rb) = s p (rb)/a = —s p (t b) = s(b)/r, 
ar 

where s p is defined as s before but now with respect to the defining 
function p; a contradiction to the definition of the infimum. 

To summarize what we know so far (*): if b G d g D, b 7^ 0, and 
Crib] •), restricted to the complex tangent hyperplane T^(dD), is not 
positive semi-definite, then tb < 1 and either tb — or if % > 0, then 
t b b G dD \ d g D. 

Now fix such a point, say again b; we may assume that b = (bi, . . . , b n ) 
with b\ = Re 61 < 0. From above we know that the segment (tbb, b] 
belongs to T b c (dD). Then the sphere Sb with center passing through 
b intersect dD transversally at b. So, there exists a neighborhood U of 
b such that P& = dD n £& fl C/& is a smooth surface of real codimension 
2 containing only C 2 -smooth non-pseudoconvex boundary points. Fix 
a t G (0, 1) and put Q\ = {c G P b : t c > t}. By (*) the set R\ = 
{t c c : c G Qt} (possible empty) contains only non C 2 -smooth boundary 
points and hence T-i 2n ^ 2 (R t b ) = 0. Note that if (M n (xk, r k )) k is a covering 
of R\ with m := inf fe \\x k \\ > 0, then (M n (y k , s k )) k is a covering of Q\ 
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where y k = [^~\\ x k an d Sk = -^r k . Hence rl 2n 2 {Ql) = 0. Therefore, 
there is a dense subset P b l of Pb such that (t, l]c G dD for any c G P b *. 
In particular, (t, l]b G clD. And because of the arbitrariness of t we 
end up with the information, that (0, l]b G dD and so G dD and 
b G Tq(8D). 

Observe that b was arbitrarily chosen. Therefore, every C 2 -boundary 
points a G d g D at which D is not pseudoconvex lie on the tangent plane 
at to dD. Fix one such a$. Then this set (of a's) is locally near a$ 
an open part of a real hypersurface which is contained in T^(dD). So 
dD is locally near ao equal to this linear real hyperplane, which implies 
that ao is a C 2 -boundary point with vanishing Levi form on T^ o (dD); 
a contradiction. □ 

Remark 2. (a) Let s be the function which was introduced during the 
proof of Theorem [6j If we replace the assumption of (^-smoothness 
by boundedness of s from below, the result remains true and the proof 
above becomes essential easier. 

(b) The proof above also shows that the C 1 -smoothness can be re- 
placed by the weaker condition that the convex hull of the tangent cone 
(in sense of Whitney) at any boundary point is of real codimension at 
least 1. 

Using an old result from [1] we also have the following consequence. 

Corollary 2. Let D C C n be a C 1 -smooth open set such that the set 
A of non C 2 -smooth boundary points of D has zero (2n — 2)-Hausdorff 
measure and it is locally contained in an analytic subset of codimen- 
sion at least 1 (any discrete set has these properties) . Then D has no 
exceptional points. 

Remark 3. Finally, let us pose the following problems. 

(1) Does an arbitrary non-pseudoconvex set DcC 3 allow a com- 
plex line I such that C 3 \ S C / (see footnote 1) ? 

(2) Does the exceptional complex line in Theorem\^have more than 
one exceptional point? 

(3) Let D C C™ be a C l -smooth non-pseudoconvex open set such 
that T-L 2n ~ 2 (dD \ d g D) = 0. Is there automatically a non- 
pseudoconvex boundary point sitting in d g D (compare with [5]) ? 
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